Evaluations on Several Smoothing Methods for Chinese Language Models
نویسندگان
چکیده
منابع مشابه
Comparison of Several Smoothing Methods in Statistical Language Model
With the development of computer technology and the appearance of huge training text corpus, the performance of language model has improved a lot recently. But its intrinsic sparse data problem still exists. This paper investigates several smoothing methods in the application of Chinese continuous speech recognition. We compare the performance of different methods, particularly in the situation...
متن کاملLanguage models and smoothing methods for information retrieval
Designing an effective retrieval model that can rank documents accurately for a given query has been a central problem in information retrieval for several decades. An optimal retrieval model that is both effective and efficient and that can learn from feedback information over time is needed. Language models are new generation of retrieval models and have been applied since the last ten years ...
متن کاملLocalized Smoothing for Multinomial Language Models
We explore a formal approach to dealing with the zero frequency problem that arises in applications of probabilistic models to language. In this report we introduce the zero frequency problem in the context of probabilistic language models, describe several popular solutions, and introduce localized smoothing, a potentially better alternative. We formulate localized smoothing as a two-step maxi...
متن کاملSmoothing methods in maximum entropy language modeling
This paper discusses various aspects of smoothing techniques in maximum entropy language modeling, a topic not sufficiently covered by previous publications. We show (1) that straightforward maximum entropy models with nested features, e.g. tri–, bi–, and unigrams, result in unsmoothed relative frequencies models; (2) that maximum entropy models with nested features and discounted feature count...
متن کاملAxiomatic Analysis of Smoothing Methods in Language Models for Pseudo-relevance Feedback by Hussein Hazimeh Thesis
Pseudo-Relevance Feedback (PRF) is an important general technique for improving retrieval effectiveness without requiring any user effort. Several state-of-the-art PRF models are based on the language modeling approach where a query language model is learned based on feedback documents. In all these models, feedback documents are represented with unigram language models smoothed with a collecti...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Information Technology Journal
سال: 2013
ISSN: 1812-5638
DOI: 10.3923/itj.2013.3685.3691